| PVE下使用OpenvSwitch实现网络绑定聚合负载均衡 | 您所在的位置：网站首页 › 网卡 brd › PVE下使用OpenvSwitch实现网络绑定聚合负载均衡 |
PVE下使用OpenvSwitch实现网络绑定聚合负载均衡
|
前由 1. 我们在企业内部实际使用中，通常需要使用vlan划分几个网段。 PVE默认使用Linux自带的网桥提供网络交换服务，在划分vlan的时候还需要修改IP路由表文件，配置稍显繁琐。 这里推荐使用专门为虚拟化设计的Open vSwitch。 Open vSwitch是一个高质量的、多层虚拟交换机，使用开源Apache2.0许可协议，由Nicira Networks开发，主要实现代码为可移植的C代码。它的目的是让大规模网络自动化可以通过编程扩展,同时仍然支持标准的管理接口和协议（例如NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag）。此外,它被设计位支持跨越多个物理服务器的分布式环境，类似于VMware的vNetwork分布式vswitch或Cisco Nexus 1000 V。Open vSwitch支持多种linux 虚拟化技术，包括Xen/XenServer， KVM和irtualBox。 https://www.openvswitch.org/ 2. 服务器通常有不止一块网卡。 很多人喜欢把几块网卡配置为主备模式，这样虽然有冗余，可是也浪费了一半的带宽。我个人更喜欢使用LACP(链路聚合控制协议)，让多条链路汇聚，既增加了带宽提高了流量，同时也保证了链路的安全冗余。 安装 # apt install openvswitch-switch -y配置 1. PVE配置 我一直不习惯使用PVE的网络配置界面，反正不管怎么，最后修改都是这一个文件：/etc/network/interfaces 下面的例子是把两块物理网卡(enp9s0 enp10s0)绑定为一块逻辑网卡(bond0，绑定模式为LACP)，然后在上面架设网关(vmbr0)，供vlan10/vlan11/vlan12通行 # cat /etc/network/interfaces auto lo iface lo inet loopback allow-vmbr1 bond0 iface bond0 inet manual ovs_bonds enp9s0 enp10s0 ovs_type OVSBond ovs_bridge vmbr1 ovs_options bond_mode=balance-slb lacp=active pre-up ( ip link set mtu 9000 dev enp9s0 && ip link set mtu 9000 dev enp10s0 ) mtu 9000 auto vmbr1 iface vmbr1 inet manual ovs_type OVSBridge ovs_ports bond0 vlan10 vlan11 vlan12 allow-vmbr1 vlan10 iface vlan10 inet static address 192.168.10.11 netmask 255.255.255.0 gateway 192.168.10.254 ovs_type OVSIntPort ovs_bridge vmbr1 ovs_options tag=10 ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif allow-vmbr1 vlan11 iface vlan11 inet static address 192.168.11.11 netmask 255.255.255.0 gateway 192.168.10.254 ovs_type OVSIntPort ovs_bridge vmbr1 ovs_options tag=11 ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif allow-vmbr1 vlan12 iface vlan12 inet static address 192.168.12.11 netmask 255.255.255.0 gateway 192.168.10.254 ovs_type OVSIntPort ovs_bridge vmbr1 ovs_options tag=12 ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif重启网络服务： # /etc/init.d/networking restart如果没有生效就重启一遍PVE服务器： # reboot2. 交换机配置 瑞捷S5750-24GT/8SFP-S 绑定示例：G0/1和G0/2，G0/3和G0/4，G0/5和G0/6 ruijie02#show run ...... interface GigabitEthernet 0/1 port-group 1 mode active ! interface GigabitEthernet 0/2 port-group 1 mode active ! interface GigabitEthernet 0/3 port-group 2 mode active ! interface GigabitEthernet 0/4 port-group 2 mode active ! interface GigabitEthernet 0/5 port-group 3 mode active ! interface GigabitEthernet 0/6 port-group 3 mode active ...... interface AggregatePort 1 switchport mode trunk switchport trunk allowed vlan remove 1-9,13-4094 ! interface AggregatePort 2 switchport mode trunk switchport trunk allowed vlan remove 1-9,13-4094 ! interface AggregatePort 3 switchport mode trunk switchport trunk allowed vlan remove 1-9,13-4094 ......测试 在PVE服务器上查看网络接口 使用linux自带的ip命令 # ip addr show 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever #下面可以看到两块网卡都属于ovs-system 2: enp9s0: mtu 9000 qdisc pfifo_fast master ovs-system state UP group default qlen 1000 link/ether xx:xx:xx:75:d2:d7 brd ff:ff:ff:ff:ff:ff inet6 fe80::325a:3aff:fe75:d2d7/64 scope link valid_lft forever preferred_lft forever 3: enp10s0: mtu 9000 qdisc pfifo_fast master ovs-system state UP group default qlen 1000 link/ether xx:xx:xx:75:d2:d8 brd ff:ff:ff:ff:ff:ff inet6 fe80::325a:3aff:fe75:d2d8/64 scope link valid_lft forever preferred_lft forever ...... #下面是虚拟出来的网络交换设备，包括bond0,网关vmbr0和vlan 6: ovs-system: mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether fa:2f:ff:73:d0:f9 brd ff:ff:ff:ff:ff:ff 7: vmbr1: mtu 9000 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether xx:xx:xx:75:d2:d8 brd ff:ff:ff:ff:ff:ff inet6 fe80::325a:3aff:fe75:d2d8/64 scope link valid_lft forever preferred_lft forever 8: bond0: mtu 9000 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether f6:af:9e:5e:5b:49 brd ff:ff:ff:ff:ff:ff inet6 fe80::f4af:9eff:fe5e:5b49/64 scope link valid_lft forever preferred_lft forever ...... 10: vlan10: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 36:1c:86:ed:a6:d3 brd ff:ff:ff:ff:ff:ff inet 192.168.10.11/24 brd 192.168.10.255 scope global vlan10 valid_lft forever preferred_lft forever inet6 fe80::341c:86ff:feed:a6d3/64 scope link valid_lft forever preferred_lft forever 11: vlan11: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether da:26:e8:18:85:35 brd ff:ff:ff:ff:ff:ff inet 192.168.11.11/24 brd 192.168.11.255 scope global vlan11 valid_lft forever preferred_lft forever inet6 fe80::d826:e8ff:fe18:8535/64 scope link valid_lft forever preferred_lft forever valid_lft forever preferred_lft forever ...... ＃下面是分配给虚拟机的网卡 14: tap103i0: mtu 9000 qdisc pfifo_fast master ovs-system state UNKNOWN group default qlen 1000 link/ether fa:4e:69:07:e9:73 brd ff:ff:ff:ff:ff:ff ......直接使用Open vSwitch工具查看： # ovs-appctl bond/show ---- bond0 ---- bond_mode: balance-slb bond may use recirculation: no, Recirc-ID : -1 bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms next rebalance: 2454 ms lacp_status: negotiated lacp_fallback_ab: false active slave mac: xx:xx:xx:75:d2:d8(enp10s0) slave enp10s0: enabled active slave may_enable: true hash 8: 3 kB load hash 38: 31 kB load hash 66: 1 kB load hash 145: 2 kB load hash 146: 7 kB load slave enp9s0: enabled may_enable: true hash 33: 171 kB load # ovs-appctl lacp/show ---- bond0 ---- status: active negotiated sys_id: xx:xx:xx:75:d2:d8 sys_priority: 65534 aggregation key: 1 lacp_time: slow slave: enp10s0: current attached port_id: 2 port_priority: 65535 may_enable: true actor sys_id: xx:xx:xx:75:d2:d8 actor sys_priority: 65534 actor port_id: 2 actor port_priority: 65535 actor key: 1 actor state: activity aggregation synchronized collecting distributing partner sys_id: xx:xx:xx:34:69:52 partner sys_priority: 32768 partner port_id: 2 partner port_priority: 32768 partner key: 1 partner state: activity aggregation synchronized collecting distributing slave: enp9s0: current attached port_id: 1 port_priority: 65535 may_enable: true actor sys_id: xx:xx:xx:75:d2:d8 actor sys_priority: 65534 actor port_id: 1 actor port_priority: 65535 actor key: 1 actor state: activity aggregation synchronized collecting distributing partner sys_id: xx:xx:xx:34:69:52 partner sys_priority: 32768 partner port_id: 1 partner port_priority: 32768 partner key: 1 partner state: activity aggregation synchronized collecting distributing我们最关心的应该还是流量是否分担到了两条链路上: # apt install nload -y # nload -m enp9s0 enp10s0 可以看到两块物理网卡都有流量，也可以到交换机上查看：  后附 1. Open vSwitch自带的命令很多，创建／删除／查看都有，可以自己多试试； 2. 华为和华三交换机(还是S5700系列，为什么各家的网络交换机取名都差不多？)的配置示例： ...... # interface Eth-Trunk1 port link-type trunk port trunk allow-pass vlan 2 to 3999 mode lacp # interface Eth-Trunk2 port link-type trunk port trunk allow-pass vlan 2 to 3999 mode lacp # interface Eth-Trunk3 port link-type trunk port trunk allow-pass vlan 2 to 3999 mode lacp ...... interface GigabitEthernet0/0/1 eth-trunk 1 # interface GigabitEthernet0/0/2 eth-trunk 1 # interface GigabitEthernet0/0/3 eth-trunk 2 # interface GigabitEthernet0/0/4 eth-trunk 2 # interface GigabitEthernet0/0/5 eth-trunk 3 # interface GigabitEthernet0/0/6 eth-trunk 3 ......3. 主备模式 如果你的交换机不支持LACP，用主备也可以的，只需要 修改绑定模式从lacp为主备 #ovs_options bond_mode=balance-slb lacp=active ovs_options bond_mode=active-backup记得交换机那边相应的取消LACP。 4. 不做绑定 即使不做任何绑定，单个网卡也可以使用Open vSwitch替换PVE自带的Linux网桥，这里就不专门讲解了。 |
【本文地址】
公司简介
联系我们